We present a data-driven stochastic optimal control framework that is derived using the path integral (PI) control approach. We find iterative control laws analytically without a priori policy parameterization based on probabilistic representation of the learned dynamics model. The proposed algorithm operates in a forward-backward sweep manner which differentiate it from other PI-related methods that perform forward sampling to find open-loop optimal controls. Our method uses significantly less sampled data to find analytic control laws compared to other approaches within the PI control family that rely on extensive sampling from given dynamics models or trials on physical systems in a model-free fashion. In addition, the learned controllers can be generalized to new tasks without re-sampling based on the compositionality theory for the linearly-solvable optimal control framework.We provide experimental results on three different systems and comparisons with state-of-the-art model-based methods to demonstrate the efficiency and generalizability of the proposed framework.

Prompt Engineering (PE) has emerged as a critical technique for guiding Large Language Models (LLMs) in solving intricate tasks. Its importance is highlighted by its potential to significantly enhance the efficiency and effectiveness of human-machine interaction. As tasks grow increasingly complex, recent advanced PE methods have extended beyond the limitations of single-round interactions to embrace multi-round interactions, which allows for a deeper and more nuanced engagement with LLMs. In this paper, we propose an optimal control framework tailored for multi-round interactions with LLMs. This framework provides a unified mathematical structure that not only systematizes the existing PE methods but also sets the stage for rigorous analytical improvements. Furthermore, we extend this framework to include PE via ensemble methods and multi-agent collaboration, thereby enlarging the scope of applicability. By adopting an optimal control perspective, we offer fresh insights into existing PE methods and highlight theoretical challenges that warrant future research. Besides, our work lays a foundation for the development of more effective and interpretable PE methods.

Practical deployments of coordinated fleets of mobile robots in different environments have revealed the benefits of maintaining small distances between robots, especially as they move at higher speeds. However, this is counter-intuitive in that as speed increases, reducing the amount of space between robots also reduces the time available to the robots to respond to sudden motion variations in surrounding robots. However, in certain examples, the benefits in performance due to traveling at closer distances can outweigh the potential instability issues, for instance, autonomous trucks on highways that optimize energy by vehicle ``drafting'' or smaller robots in cluttered environments that need to maintain close, line of sight communication, etc. To achieve this kind of closely coordinated fleet behavior, this work introduces a model predictive optimal control framework that directly takes non-linear dynamics of the vehicles in the fleet into account while planning motions for each robot. The robots are able to follow each other closely at high speeds by proactively making predictions and reactively biasing their responses based on state information from the adjacent robots. This control framework is naturally decentralized and, as such, is able to apply to an arbitrary number of robots without any additional computational burden. We show that our approach is able to achieve lower inter-robot distances at higher speeds compared to existing controllers. We demonstrate the success of our approach through simulated and hardware results on mobile ground robots.

We present a data-driven stochastic optimal control framework that is derived using the path integral (PI) control approach. We find iterative control laws analytically without a priori policy parameterization based on probabilistic representation of the learned dynamics model. The proposed algorithm operates in a forward-backward sweep manner which differentiate it from other PI-related methods that perform forward sampling to find open-loop optimal controls. Our method uses significantly less sampled data to find analytic control laws compared to other approaches within the PI control family that rely on extensive sampling from given dynamics models or trials on physical systems in a model-free fashion. In addition, the learned controllers can be generalized to new tasks without re-sampling based on the compositionality theory for the linearly-solvable optimal control framework.We provide experimental results on three different systems and comparisons with state-of-the-art model-based methods to demonstrate the efficiency and generalizability of the proposed framework.